Method and an apparatus for decoding/encoding a video signal by performing illumination compensation

ABSTRACT

A method for recovering transmission errors, comprising: receiving a data packet comprising an error detection code associated to data contained in the packet, wherein the data associated to the error detection code comprises primary data and secondary data, checking the error detection code of the received packet to detect an erroneous state of the associated data, when the erroneous state is detected, determining a finite set of candidate values for the primary data and, for each values of the set: determining a marginal likelihood of the candidate value as a function of the error detection code of the received packet, determining a first correlation between the primary data of the received packet and the candidate value, and selecting a corrected value for the primary data among the set of candidate values as a function of said marginal likelihoods and said first correlations.

This application is a National Phase entry of PCT Application numberPCT/KR2008/001631 filed on Mar. 24, 2008, which claims priority under 35U.S.C. §§119(e), 120 and 365(c) to U.S. Provisional Application Nos.60/896,638, 60/929,220, 60/947,648 and 60/979,868 filed on Mar. 23,2007, Jun. 18, 2007, Jul. 2, 2007 and Oct. 15, 2007, respectively.

TECHNICAL FIELD

The present invention relates to coding of a video signal.

BACKGROUND ART

Compression coding means a series of signal processing techniques fortransmitting digitalized information via a communication circuit orstoring the digitalized information in a form suitable for a storagemedium. As targets of compression coding, there are audio, video,characters, etc. In particular, a technique for performing compressioncoding on video is called video sequence compression. A video sequenceis generally characterized in having spatial redundancy or temporalredundancy.

DISCLOSURE OF THE INVENTION Technical Problem

Accordingly, the present invention is directed to a method and apparatusfor decoding/encoding a video signal that can substantially enhanceefficiency in coding the video signal.

Technical Solution

An object of the present invention is to enhance coding efficiency ofvideo sequence using inter-block or inter-view correlation effectively.

Another object of the present invention is to efficiently compensate forillumination difference between views in multi-view sequence.

Another object of the present invention is to efficiently transmitinter-view reference information indicating inter-view dependency.

Another object of the present invention is to enhance coding efficiencyof a video signal using depth information.

Another object of the present invention is to enhance coding efficiencyof a video signal or provide a user with various services using camerainformation.

A further object of the present invention is to provide a decodingmethod and apparatus, by which video sequence data can be efficientlydecoded.

ADVANTAGEOUS EFFECTS

According to the present invention, signal processing efficiency can beenhanced by predicting motion information using temporal and spatialcorrelations of a video sequence. More precise prediction is enabled bypredicting coding information of a current block using codinginformation of a picture having high correlation with the current block,whereby a transmitted error is reduced to perform efficient coding. Evenif motion information of a current block is not transmitted, it is ableto calculate motion information very similar to that of the currentblock. Hence, a restoration rate is enhanced. It is able to improvecoding efficiency and a sequence reconstruction rate using depthinformation. It is able to enhance coding efficiency using camerainformation. And, it is also able to provide various services to a user.

In the present invention, an offset value of a current block ispredicted using information of a neighbor block and a correspondingdifference value is transmitted. Hence, it is able to minimizeinformation that should be transmitted for illumination compensation(hereinafter abbreviated IC). In case of predictive coding using atleast two reference blocks, more efficient coding is enabled by applyingoffset value and flag information by at least one of various methods.When the offset value of the current block is predicted, more accurateprediction can be performed by checking whether a reference index of thecurrent block is identical to that of the neighbor block. It is able tominimize information, which should be transmitted, in a manner ofpredicting flag information indicating whether to perform illuminationcompensation of a current block and then transmitting a correspondingdifference value only. Likewise, more accurate prediction can beperformed by checking whether a reference index of the current block isidentical to that of a neighbor block.

DESCRIPTION OF DRAWINGS

The accompanying drawings, which are included to provide a furtherunderstanding of the invention and are incorporated in and constitute apart of this specification, illustrate embodiments of the invention andtogether with the description serve to explain the principles of theinvention.

In the drawings:

FIG. 1 is a schematic block diagram of a video signal decoding apparatusaccording to an embodiment of the present invention;

FIG. 2 is a diagram of configuration informations on a multi-viewsequence that can be added to a multi-view sequence coded bit streamaccording to an embodiment of the present invention;

FIG. 3 is a diagram to explain a process for obtaining IC differenceinformation of a current block according to one embodiment of thepresent invention;

FIG. 4 and FIG. 5 are diagrams to explain a motion vector predictingmethod by considering illumination compensation according to anembodiment of the present invention;

FIG. 6 is a diagram to explain a coding method using a depth mapaccording to an embodiment of the present invention;

FIGS. 7 to 14 are diagrams of syntaxes for describing variousapplication examples that use camera information according to anembodiment of the present invention;

FIG. 15 is a diagram of an overall prediction structure of a multi-viewsequence signal according to an embodiment of the present invention toexplain a concept of an inter-view picture group; and

FIGS. 16 to 20 are diagrams of various syntaxes for describinginter-view reference information according to an embodiment of thepresent invention.

BEST MODE

Additional features and advantages of the invention will be set forth inthe description which follows, and in part will be apparent from thedescription, or may be learned by practice of the invention. Theobjectives and other advantages of the invention will be realized andattained by the structure particularly pointed out in the writtendescription and claims thereof as well as the appended drawings.

To achieve these and other advantages and in accordance with the purposeof the present invention, as embodied and broadly described, a method ofdecoding a video signal includes obtaining IC difference information ofa neighbor block of a current block, when information associated withillumination compensation for the current block is not available in abitstream, deriving IC difference information of the current block usingthe IC difference information of the neighbor block, deriving an IC flagindicating whether illumination compensation is performed on the currentblock, based on the IC difference information of the current block, andperforming the illumination compensation on the current block, based onthe IC difference information of the current block and the IC flag.

Preferably, when the IC difference information of the current block is0, the IC flag of the current block is derived into 0.

Preferably, when the IC difference information of the current block isnot 0, the IC flag of the current block is derived into 1.

Preferably, the IC difference information of the current block isderived based on whether a reference index of the current block isidentical to that of the neighbor block.

More preferably, when there exists one neighbor block having thereference index identical to that of the current block, the ICdifference information of the current block is set to the IC differenceinformation of the neighbor block.

In this case, the neighbor block is checked in order of upper, left,upper-right and upper-left blocks of the current block.

More preferably, when there exist three neighbor blocks, each having thereference index identical to that of the current block, the ICdifference information of the current block is set to a median value ofIC difference informations of the three neighbor blocks.

Preferably, the video signal is received as a broadcast signal.

Preferably, the video signal is received via digital medium.

To further achieve these and other advantages and in accordance with thepurpose of the present invention, a computer-readable medium includes aprogram for executing the present invention. And, the program isrecorded in the computer-readable medium.

To further achieve these and other advantages and in accordance with thepurpose of the present invention, an apparatus for decoding a videosignal includes an IC difference prediction unit deriving IC differenceinformation of a current block using an obtained IC differenceinformation of a neighbor block, when information associated withillumination compensation for the current block is not available in abitstream; and

an IC unit deriving an IC flag indicating whether illuminationcompensation is performed on the current block based on the ICdifference information of the current block, and performing theillumination compensation on the current block based on the ICdifference information of the current block and the IC flag.

It is to be understood that both the foregoing general description andthe following detailed description are exemplary and explanatory and areintended to provide further explanation of the invention as claimed.

MODE FOR INVENTION

Reference will now be made in detail to the preferred embodiments of thepresent invention, examples of which are illustrated in the accompanyingdrawings.

First of all, compression coding of video signal data considers spatialredundancy, spatial redundancy, scalable redundancy, and inter-viewredundancy. And, compression coding is enabled by considering inter-viewexisting mutual redundancy in the course of the compression coding.Compression coding scheme, which takes inter-view redundancy intoconsideration, is just an embodiment of the present invention. And, thetechnical idea of the present invention is applicable to temporalredundancy, scalable redundancy, and the like. In this disclosure,coding can include both concepts of encoding and decoding. And, codingcan be flexibly interpreted to correspond to the technical idea andscope of the present invention.

Looking into a bit sequence configuration of a video signal, thereexists a separate layer structure called a NAL (network abstractionlayer) between a VCL (video coding layer) dealing with a moving pictureencoding process itself and a lower system that transmits and storesencoded information. An output from an encoding process is VCL data andis mapped by NAL unit prior to transmit or storage. Each NAL unitincludes compressed video data or RBSP (raw byte sequence payload:result data of moving picture compression) that is the datacorresponding to header information.

The NAL unit basically includes two parts, a NAL header and an RBSP. TheNAL header includes flag information (nal_ref_idc) indicating whether aslice as a reference picture of the NAL unit is included and anidentifier (nal_unit_type) indicating a type of the NAL unit. Compressedoriginal data is stored in the RBSP. And, RBSP trailing bit is added toa last portion of the RBSP to represent a length of the RESP as an 8-bitmultiplication. As the types of the NAL unit, there are IDR(instantaneous decoding refresh) picture, SPS (sequence parameter set),PPS (picture parameter set), SEI (supplemental enhancement information),and the like.

In the standardization, requirements for various profiles and levels areset to enable implementation of target product with an appropriate cost.In this case, a decoder should meet the requirements determinedaccording to the corresponding profile and level. Thus, two concepts,‘profile’ and ‘level’ are defined to indicate a function or parameterfor representing how far the decoder can cope with a range of acompressed sequence. And, a profile identifier (profile_idc) canidentify that a bit stream is based on a prescribed profile. The profileidentifier means a flag indicating a profile on which a bit stream isbased. For instance, in H.264/AVC, if a profile identifier is 66, itmeans that a bit stream is based on a baseline profile. If a profileidentifier is 77, it means that a bit stream is based on a main profile.If a profile identifier is 88, it means that a bit stream is based on anextended profile. Moreover, the profile identifier can be included in asequence parameter set.

So, in order to handle a multi-view sequence, it needs to be identifiedwhether an inputted bit stream is a multi-view profile. If the inputtedbit stream is the multi-view profile, it is necessary to add syntax toenable at least one additional information for multi-view to betransmitted. In this case, the multi-view profile indicates a profilemode for handling multi-view video as an additional technique ofH.264/AVC. In MVC, it may be more efficient to add syntax as additionalinformation for an MVC mode rather than unconditional syntax. Forinstance, when a profile identifier of AVC indicates a multi-viewprofile, if information for a multi-view sequence is added, it is ableto enhance encoding efficiency.

Sequence parameter set indicates header information containinginformation crossing over encoding of an overall sequence such as aprofile, a level, and the like. A whole compressed moving picture, i.e.,a sequence should start from a sequence header. So, a sequence parameterset corresponding to header information should arrive at a decoderbefore the data referring to the parameter set arrives. Namely, thesequence parameter set RBSP plays a role as the header information forthe result data of the moving picture compression. Once a bit stream isinputted, a profile identifier preferentially identifies that theinputted bit stream is based on which one of a plurality of profiles.So, by adding a part for determining whether an inputted bit streamrelates to a multi-view profile (e.g., ‘If(profile_idc==MULTI_VIEW_PROFILE)’) to syntax, it is determined whetherthe inputted bit stream relates to the multi-view profile. Various kindsof configuration information can be added only if the inputted bitstream is approved as relating to the multi-view profile. For instance,it is able to add a total number of views, a number of inter-viewreference pictures, a view identification number of an inter-viewreference picture, and the like. And, a decoded picture buffer can usevarious kinds of informations on an interview reference picture toconstruct and manage a reference picture list.

FIG. 1 is a schematic block diagram of an apparatus for decoding a videosignal according to the present invention.

Referring to FIG. 1, the decoding apparatus includes a parsing unit 100,an entropy decoding unit 200, an inverse quantization/inverse transformunit 300, an intra prediction unit 400, a deblocking filter unit 500, adecoded picture buffer unit 600, an inter prediction unit 700, and thelike. And, the inter prediction unit 700 can include an IC differenceprediction unit 710, an IC (illumination compensation) unit 720, amotion compensation unit 730, and the like.

The parsing unit 100 carries out parsing by NAL unit to decode areceived video sequence. In general, at least one sequence parameter setand at least one picture parameter set are transferred to a decoderbefore a slice header and slice data are decoded. In this case, variouskinds of configuration informations can be included in a NAL header areaor an extension area of a NAL header. Since MVC is an additional schemefor a conventional AVC scheme, it may be more efficient to add variousconfiguration informations in case of an MVC bit stream only rather thanunconditional addition. For instance, it is able to add flag informationfor identifying a presence or non-presence of an MVC bit stream in theNAL header area or the extension area of the NAL header. Only if aninputted bit stream is a multi-view sequence coded bit stream accordingto the flag information, it is able to add configuration informationsfor a multi-view sequence. For instance, the configuration informationscan include view identification information, inter-view picture groupidentification information, inter-view prediction flag information,temporal level information, priority identification information,identification information indicating whether it is an instantaneousdecoded picture for a view, and the like. They will be explained indetail with reference to FIG. 2.

The entropy decoding unit 200 carries out entropy decoding on a parsedbit stream and a coefficient of each macroblock, a motion vector, andthe like are then extracted. The inverse quantization/inverse transformunit 300 obtains a coefficient value transformed by multiplying areceived quantized value by a predetermined constant and then transformsthe coefficient value inversely to reconstruct a pixel value. Using thereconstructed pixel value, the intra prediction unit 400 performsintra-screen prediction from a decoded sample within a current picture.Meanwhile, the deblocking filter unit 500 is applied to each codedmacroblock to reduce block distortion. A filter smoothens a block edgeto enhance an image quality of a decoded frame. Selection of a filteringprocess depends on boundary strength and gradient of an image samplearound a boundary. Pictures through filtering are outputted or stored inthe decoded picture buffer unit 600 to be used as reference pictures.

The decoded picture buffer unit 600 plays a role in storing or openingthe previously coded pictures to perform inter-picture prediction. Inthis case, to store the pictures in the decoded picture buffer unit 600or to open the pictures, ‘frame_num’ of each picture and POC (pictureorder count) are used. So, in MVC, since there exist pictures in a viewdifferent from that of a current picture exists among the previouslycoded pictures, in order to use these pictures as reference pictures,view information for identifying a picture is usable together with the‘frame_num’ and the POC.

The decoded picture buffer unit 600 can use information on view inconstructing the reference picture list for the inter-view prediction.For instance, inter-view reference information can be used. Inter-viewreference information means information used to indicate an inter-viewdependent relation. For instance, there can be a total number of views,a view identification number, a number of inter-view reference pictures,a view identification number of an inter-view reference picture, and thelike.

The decoded picture buffer unit 600 manages reference pictures torealize inter-picture prediction more flexibly. For instance, a memorymanagement control operation method and a sliding window method areusable. This is to manage a reference picture memory and a non-referencepicture memory by unifying the memories into one memory and realizeefficient memory management with a small memory. In multi-view videocoding, since pictures in a view direction have the same picture ordercount, information for identifying a view of each of the pictures isusable in marking them. And, reference pictures managed in the abovemanner can be used by the inter prediction unit 700.

Inter-coded macroblock can be divided into macroblock partitions. And,each of the macroblock partitions can be predicted from one or tworeference pictures. The inter prediction unit 700 includes the ICdifference prediction unit 710, the IC unit 720, the motion compensationunit 730, and the like.

In case that an inputted bitstream corresponds to a multi-view sequence,since the respective view sequences are captured by different cameras,respectively, an illumination difference is generated due to internaland external factors of the cameras. To prevent this, the IC unit 720performs illumination compensation (IC). In performing illuminationcompensation, it is able to use flag information indicating whether toperform illumination compensation on a prescribed layer of a videosignal. For instance, it is able to perform illumination compensationusing flag information indicating whether to perform illuminationcompensation on a corresponding slice or macroblock. In performingillumination compensation using the flag information, it is applicableto various kinds of macroblock types (e.g., inter 16×16 mode, B-skipmode, direct mode, etc.).

In performing illumination compensation, it is able to use informationof a neighbor block, information of a block in a view different fromthat of a current block or IC difference information of the currentblock to reconstruct the current block. In case that a current blockrefers to neighbor blocks in different view, it is able to performillumination compensation using information on a reference picture listfor inter-view prediction which is stored in the decoded picture bufferunit 600. In this case, the OC difference value of the current block maymean a difference between an average pixel value of the current blockand an average pixel value of a corresponding reference block. Forexample of using the IC difference information, an IC differencepredictive value of the current block is obtained using neighbor blocksof the current block and it is able to use an IC difference residual (ICoffset residual) that is a difference value between the IC differenceinformation and the IC difference predictive value. Hence, a decoder isable to reconstruct the IC difference information of the current blockusing the IC difference residual and the IC difference predictive value.

In obtaining an IC difference predictive value of a current block, it isable to use information of a neighbor block. For instance, it is able topredict IC difference information of a current block using IC differenceinformation of a neighbor block. Prior to this, it is checked whether areference index of the current block is identical to that of theneighbor block. According to a result of the check, it is able todetermine whether to use a prescribed block or a prescribed value.

It is able to predict coding information of a current block using codinginformation correlation of a view direction. For instance, codinginformation can include illumination compensation information,predictive direction information, partition information or the like. Inparticular, in order to predict illumination compensation information ofa current block, it is able to use illumination compensation of blocksneighboring to the current block. And, it is also able to useillumination compensation information of a block of a picture whichcorresponds to the current block and exists in the same view of thecurrent block.

The motion compensation unit 730 compensates for a motion of a currentblock using informations transmitted from the entropy decoding unit 200.Motion vectors of blocks neighbor to the current block are extractedfrom a video signal and a motion vector of the current block are thenobtained. And, the motion of the current block is compensated using theobtained motion vector predicted value and a differential vectorextracted from the video signal. And, it is able to perform the motioncompensation using one reference picture or a plurality of pictures. Inmulti-view video coding, in case that a current picture refers topictures in different views, it is able to perform motion compensationusing information for the inter-view prediction reference picture liststored in the decoded picture buffer unit 600. And, it is also able toperform motion compensation using view information for identifying aview of the corresponding picture.

Direct prediction mode is an encoding mode for predicting motioninformation of a current block from motion information of an encodedblock. Since this method is able to save a count of bits required fordecoding the motion information, compression efficiency is enhanced. Forinstance, a temporal direct mode predicts motion information of acurrent block using motion information correlation in a temporaldirection. The temporal direct mode is effective when a speed of themotion in a sequence containing different motions is constant. In casethat the temporal direct mode is used for multi-view video coding,inter-view motion vector should be taken into consideration.

For another example of the direct prediction mode, a spatial direct modepredicts motion information of a current block using motion informationcorrelation in a spatial direction. The spatial direct mode is effectivewhen a speed of motion varies in a sequence containing the same motions.Within a reference picture having a smallest reference number in areverse direction reference picture list (List 1) of a current picture,it is able to predict motion information of the current picture usingmotion information of a block co-located with the current block. Yet, inmulti-view video coding, the reference picture may exist in a viewdifferent from that of the current picture. In this case, variousembodiments are usable in applying the spatial direct mode.

In video signal coding, it is able to use depth information for aspecific application or other purposes. The depth information may meanthe information capable of indicating inter-view disparity difference.For instance, it is able to obtain a disparity vector through inter-viewprediction. And, the obtained disparity vector should be transmitted toa decoding device for disparity compensation of a current block. Yet, incase that a depth map is found and then transmitted to a decodingdevice, it is able to derive the disparity vector from the depth map (ordisparity map) without transmitting the disparity vector to the decodingdevice. Depth map can be transmitted together with a motion vector or adisparity vector. In this case, the depth map may mean depth informationfor each predetermined unit. For instance, the predetermined unit maycorrespond to a pixel unit or a block unit. If so, it is advantageousthat the number of bits of the depth information, which should betransmitted to the decoding device, can be lowered. It is able toreconstruct a virtual view between two views neighboring to each otherusing the depth map. By deriving a disparity vector from the depth map,it is able to provide a new disparity compensation method. Thus, in caseof using a picture in a different view in the process for deriving adisparity vector from the depth map, view information for identifying aview of picture can be used. This will be explained in detail withreference to FIG. 6 later.

The inter-predicted pictures and the intra-predicted pictures by theabove-explained processes are selected according to a prediction mode toreconstruct a current picture.

FIG. 2 is a diagram of configuration informations on a multi-viewsequence addable to a multi-view sequence coded bit stream according toone embodiment of the present invention.

FIG. 2 shows an example of a NAL-unit configuration to whichconfiguration informations on a multi-view sequence can be added. NALunit can mainly include NAL unit header and RBSP (raw byte sequencepayload: result data of moving picture compression). And, the NAL unitheader can include identification information (nal_ref_idc) indicatingwhether the NAL unit includes a slice of a reference picture andinformation (nal_unit_type) indicating a type of the NAL unit. And, anextension area of the NAL unit header can be limitedly included. Forinstance, if the information indicating the type of the NAL unit isassociated with scalable video coding or indicates a prefix NAL unit,the NAL unit is able to include an extension area of the NAL unitheader. In particular, if the nal_unit_type=20 or 14, the NAL unit isable to include the extension area of the NAL unit header. And,configuration informations for a multi-view sequence can be added to theextension area of the NAL unit header according to flag information(svc_mvc_flag) capable of identifying whether it is MVC bit stream.

For another instance, if the information indicating the type of the NALunit is information indicating a sequence parameter set, the RBSP caninclude information on the sequence parameter set. In particular, ifnal_unit_type=7, the RBSP can include information for a sequenceparameter set. In this case, the sequence parameter set can include anextension area of the sequence parameter set according to profileinformation. For example, if profile information (profile_idc) is aprofile relevant to multi-view video coding, the sequence parameter setcan include an extension area of the sequence parameter set.Alternatively, a subset sequence parameter set can include an extensionarea of a sequence parameter set according to profile information. Theextension area of the sequence parameter set can include inter-viewreference information indicating inter-view dependency.

Various configuration informations on a multi-view sequence, e.g.,configuration informations that can be included in an extension area ofNAL unit header or configuration informations that can be included in anextension area of a sequence parameter set are explained in detail asfollows.

First of all, view identification information means information fordiscriminating a picture in a current view from a picture in a differentview. In coding a video sequence signal, POC (picture order count) and‘frame_num’ are used to identify each picture. In case of a multi-viewvideo sequence, inter-view prediction is carried out. So, identificationinformation to discriminate a picture in a present view from a picturein another view is needed. Thus, it is necessary to define viewidentification information for identifying a view of a picture. The viewidentification information can be obtained from a header area of a videosignal. For instance, the header area can be a NAL header area, anextension area of a NAL header, or a slice header area. Information on apicture in a view different from that of a current picture is obtainedusing the view identification information and it is able to decode thevideo signal using the information on the picture in the different view.

The view identification information is applicable to an overallencoding/decoding process of the video signal. For instance, viewidentification information can be used to indicate inter-viewdependency. Number information of inter-view reference pictures, viewidentification information of an inter-view reference picture and thelike may be needed to indicate the inter-view dependency. Like thenumber information of the inter-view reference pictures or the viewidentification information of the inter-view reference picture,information used to indicate the inter-view dependency shall be namedinter-view reference information. In this case, the view identificationinformation can be used to indicate the view identification informationof the inter-view reference picture. The inter-view reference picturemay mean a reference picture used in performing inter-view prediction ona current picture. And, the view identification information can beintactly applied to multi-view video coding using ‘frame_num’ thatconsiders a view instead of considering a specific view identifier.

Inter-view picture group identification information means informationcapable of identifying whether a coded picture of a current NAL unit isan inter-view picture group. In this case, the inter-view picture groupmeans a coded picture that only refers to a slice that all slices existin a frame on a same time zone. For instance, it means a coded picturethat refers to a slice in a different view only but does not refer to aslice in a current view. In decoding a multi-view sequence, aninter-view random access may be possible. For inter-view prediction,inter-view reference information is necessary. In obtaining theinter-view reference information, inter-view picture groupidentification information is usable. For instance, if a current picturecorresponds to an inter-view picture group, inter-view referenceinformation on the inter-view picture group can be obtained. If acurrent picture corresponds to a non-inter-view picture group,inter-view reference information on the non-inter-view picture group canbe obtained.

Thus, in case that inter-view reference information is obtained based oninter-view picture group identification information, it is able toperform inter-view random access more efficiently. This is becauseinter-view reference relation between pictures in an inter-view picturegroup can differ from that in a non-inter-view picture group. And, incase of an inter-view picture group, pictures in a plurality of viewscan be referred to. For instance, a picture of a virtual view isgenerated from pictures in a plurality of views and it is then able topredict a current picture using the picture of the virtual view.

In constructing a reference picture list, the inter-view picture groupidentification information can be used. In this case, the referencepicture list can include a reference picture list for inter-viewprediction. And, the reference picture list for the inter-viewprediction can be added to the reference picture list. For instance, incase of initializing a reference picture list or modifying the referencepicture list, the inter-view picture group identification informationcan be used. And, it can be also used to manage the added referencepictures for the inter-view prediction. For instance, by dividing thereference pictures into an inter-view picture group and a non-inter-viewpicture group, it is able to make a mark indicating that referencepictures failing to be used in performing inter-view prediction shallnot be used. And, the inter-view picture group identificationinformation is applicable to a hypothetical reference decoder.

Inter-view prediction flag information means information indicatingwhether a coded picture of a current NAL unit is used for inter-viewprediction. The inter-view prediction flag information is usable for apart where temporal prediction or inter-view prediction is performed. Inthis case, identification information indicating whether NAL unitincludes a slice of a reference picture can be used together. Forinstance, although a current NAL unit fails to include a slice of areference picture according to the identification information, if it isused for inter-view prediction, the current NAL unit can be a referencepicture used for inter-view prediction only. According to theidentification information, if a current NAL unit includes a slice of areference picture and used for inter-view prediction, the current NALunit can be used for temporal prediction and inter-view prediction. IfNAL unit fails to include a slice of a reference picture according tothe identification information, it can be stored in a decoded picturebuffer. This is because, in case that a coded picture of a current NALunit is used for inter-view prediction according to the inter-viewprediction flag information, it needs to be stored.

Aside from a case of using both of the flag information and theidentification information together, one identification information canindicate whether a coded picture of a current NAL unit is used fortemporal prediction or/and inter-view prediction.

Temporal level information means information on a hierarchical structureto provide temporal scalability from a video signal. Though the temporallevel information, it is able to provide a user with a sequence onvarious time zones.

Priority identification information means information capable ofidentifying a priority of NAL unit. It is able to provide viewscalability using the priority identification information. For example,it is able to define view level information using the priorityidentification information. In this case, view level information meansinformation on a hierarchical structure for providing view scalabilityfrom a video signal. In a multi-view video sequence, it is necessary todefine a level for a time and a level for a view to provide a user withvarious temporal and view sequences. In case of defining the above levelinformation, it is able to use temporal scalability and viewscalability. Hence, a user is able to view a sequence at a specific timeand view only or a sequence according to another condition forrestriction only. The level information can be set differently invarious ways according to its referential condition. For instance, thelevel information can be set different according to camera location orcamera alignment. And, the level information can be determined byconsidering view dependency. For instance, a level for a view having aninter-view picture group of I picture is set to 0, a level for a viewhaving an inter-view picture group of P picture is set to 1, and a levelfor a view having an inter-view picture group of B picture is set to 2.Thus, the level value can be assigned to the priority identificationinformation. Moreover, the level information can be randomly set withoutbeing based on a special reference.

In the following description, various embodiments for providing anefficient decoding method of a video signal are explained.

FIG. 3 is a diagram to explain a process for obtaining IC differenceinformation of a current block according to one embodiment of thepresent invention.

In comparing similarity between a current block and a candidatereference block, an illumination difference between the two blocksshould be taken into consideration. To compensate for the illuminationdifference (illumination change), new motion estimation and motioncompensation are carried out. New SAD can be found using Formula 1.

$\begin{matrix}{{M_{curr} = {\frac{1}{S \times T}{\sum\limits_{i - m}^{m + S - 1}\;{\sum\limits_{j - n}^{m + T - 1}\;{f\left( {i,j} \right)}}}}}{{M_{ref}\left( {p,q} \right)} = {\frac{1}{S \times T}{\sum\limits_{i - p}^{p + S - 1}\;{\sum\limits_{j - q}^{q + T - 1}\;{r\left( {i,j} \right)}}}}}} & \left\lbrack {{Formula}\mspace{20mu} 1} \right\rbrack \\{{{NewSAD}\left( {x,y} \right)} = {\sum\limits_{i - m}^{m + S - 1}\;{\sum\limits_{j - n}^{m + T - 1}\;{{\left\{ {{f\left( {i,j} \right)} - M_{curr}} \right\} - \left\{ {{r\left( {{i + x},{j + y}} \right)} - {M_{ref}\left( {{m + x},{n + y}} \right)}} \right\}}}}}} & \left\lbrack {{Formula}\mspace{20mu} 2} \right\rbrack\end{matrix}$

In this case, M_(curr) indicates an average pixel value of a currentblock, M_(ref) indicates an average pixel value of a reference block,f(i,j) indicates a pixel value of a current block, and r(i+x, j+y)indicates a pixel value of a reference block. By performing motionestimation based on new SAD shown in Formula 2, it is able to obtain anaverage pixel value difference between the current block and thereference block. And, the obtained average pixel value difference isnamed IC difference information (IC_offset).

In case of performing motion estimation having illumination compensationapplied thereto, IC difference information and a motion vector areobtained. And, the illumination compensation is carried out by Formula 3using the IC difference information and the motion vector.

$\begin{matrix}\begin{matrix}{{{NewR}\left( {i,j} \right)} = {\left\{ {{f\left( {i,j} \right)} - M_{curr}} \right\} - \left\{ {{r\left( {{i + x^{\prime}},{j + y^{\prime}}} \right)} -} \right.}} \\\left. {M_{ref}\left( {{m + x^{\prime}},{n + y^{\prime}}} \right)} \right\} \\{= {\left\{ {{f\left( {i,j} \right)} - {r\left( {{i + x^{\prime}},{j + y^{\prime}}} \right)}} \right\} - M_{curr} -}} \\\left. {M_{ref}\left( {{m + x^{\prime}},{n + y^{\prime}}} \right)} \right\} \\{= {\left\{ {{f\left( {i,j} \right)} - {r\left( {{i + x^{\prime}},{j + y^{\prime}}} \right)}} \right\} - {IC\_ offset}}}\end{matrix} & \left\lbrack {{Formula}\mspace{20mu} 3} \right\rbrack\end{matrix}$

In Formula 3, NewR(i,j) indicates a residual value resulting fromexecution of illumination compensation and (x′,y′) indicates a motionvector.

IC difference information (M_(curr)−M_(ref)) should be transmitted to adecoding unit. And, the illumination compensation in the decoding unitis carried out as follows.

$\begin{matrix}\begin{matrix}{{f^{\prime}\left( {i,j} \right)} = {\left\{ {{{NewR}^{''}\left( {x^{\prime},y^{\prime},i,j} \right)} + {r\left( {{i + x^{\prime}},{j + y^{\prime}}} \right)}} \right\} +}} \\{\left\{ {M_{curr} - {M_{ref}\left( {{m + x^{\prime}},{n + y^{\prime}}} \right)}} \right\}} \\{= {\left\{ {{{NewR}^{''}\left( {x^{\prime},y^{\prime},i,j} \right)} + {r\left( {{i + x^{\prime}},{j + y^{\prime}}} \right)}} \right\} +}} \\{IC\_ offset}\end{matrix} & \left\lbrack {{Formula}\mspace{20mu} 4} \right\rbrack\end{matrix}$

In Formula 4, NewR″(i,j) indicates a reconstructed andillumination-compensated residual value and f′(I,j) indicates a pixelvalue of a reconstructed current block.

In order to reconstruct a current block, IC difference information hasto be transmitted to a decoding unit. And, the IC difference informationcan be predicted from information of neighbor blocks. To further reducethe number of bits to code the IC difference information, it is able tosend an IC difference residual (illumination compensated residualsignal) (RIC_offset) only. This can be represented as Formula 5.RIC_offset=IC_offset−predIC_offset  [Formula 5]

FIG. 4 and FIG. 5 are diagrams to explain a motion vector predictingmethod by considering illumination compensation according to anembodiment of the present invention.

A process for performing illumination compensation using flaginformation indicating whether to perform illumination compensation of acurrent block and IC difference information of the current blockaccording to an embodiment of the present invention is explained asfollows.

First of all, to perform illumination compensation, a decoding unitextracts flag information and IC difference information of neighborblocks of a current block, reference indexes of corresponding referenceblocks of the current and neighbor blocks and the like from a videosignal, and is then able to obtain an IC difference predictive value ofthe current block using the informations. And, an IC difference residualindicating a difference value between IC difference information of thecurrent block and the IC difference predictive value is obtained. The ICdifference information of the current block can be reconstructed usingthe IC difference residual value and the IC difference predictive value.In this case, in reconstructing the IC difference information of thecurrent block, it is able to use an IC flag (IC_flag) indicating whetherto perform illumination compensation on the current block.

In the first place, an IC flag indicating whether to performillumination compensation on a current block can be obtained from avideo signal. When the illumination compensation is performed accordingto the flag information, it is able to reconstruct IC differenceinformation of the current block indicating a difference between anaverage pixel value of the current block and an average pixel value of areference block. Like this, the illumination compensation techniquecodes a difference value of average pixel values of blocks belonging todifferent pictures. In case that a flag indicating whether to apply theillumination compensation technique is used for each block, when acorresponding block belongs to a P slice, a single flag information anda single IC difference information are just encoded/decoded. Yet, when acorresponding block belongs to a B slice, several methods are available.

In FIG. 4, ‘C’ indicates a current block (C), ‘N’ indicates a blockneighboring to the current block C, ‘R’ indicates a block referred to bythe current block C, and ‘S’ indicates a block referred to by the blockN neighboring to the current block C. Moreover, ‘m_(c)’ indicates anaverage pixel value of a current block and ‘m_(r)’ indicates an averagepixel value of a block referred to by the current block. Assuming thatIC difference information of the current block C is set to ‘IC_offset’,it results in ‘IC_offset=m_(c)−m_(r)’. Likewise, assuming that ICdifference information of the neighbor block N is set to‘IC_offset_pred’, an encoding unit is just able to send an IC differenceresidual (RIC_offset) indicating a difference value between the ICdifference information (IC_offset) of the current block and the ICdifference information (IC_offset_pred) of the neighbor block instead oftransmitting the value intact to reconstruct the IC differenceinformation ‘IC_offset’ of the current block C. In this case, the ICdifference residual (RIC_offset) can be represented as Formula 5. Ingenerating an IC difference predictive value of the current block fromflag information or IC difference information of neighbor block, variousmethods are applicable. For instance, it is able to use information of asingle neighbor block or information of at least two neighbor blocks. Incase of using the information of the at least two neighbor blocks, it isable to use an average or a median. Thus, if a current block is codedusing a single reference block only, it is able to perform illuminationcompensation using a single IC difference information and a single flaginformation.

Yet, in case that the corresponding block belongs to the B slice, i.e.,if a current block is coded using at least two reference blocks, variousmethods are available. For instance, in FIG. 5, assume that ‘C’indicates a current block C, ‘N’ indicates a block neighboring to thecurrent block C, ‘R0’ indicates a reference block in a reference picture1 of List 0 referred to by the current block, and ‘S0’ indicates areference block in reference picture list 1 of List 0 referred to by theneighbor block. Assume that ‘R1’ indicates a reference block inreference picture 3 of List 1 referred to by the current block and ‘S0’indicates a reference block in reference picture 3 of List 1 referred toby the neighbor block. In this case, since flag information and ICdifference information of the current block exists for each referenceblock, there exist two values each. Hence, in using the flag informationand the IC difference information, it is able to use at least one ofthem each.

A process for performing illumination compensation for a current blockaccording to an embodiment of the present invention is explained asfollows.

In case that an IC flag of a current block is 0, illuminationcompensation for the current block is not performed. In case that the ICflag is 1, a process for reconstructing IC difference information of thecurrent block is performed. In this case, in obtaining a predictivevalue of the current block, information of neighbor block is available.

For instance, it is able to predict IC difference information of thecurrent block using the IC difference information of the neighbor block.Prior to this, it is checked whether a reference index of the currentblock is identical to that of the neighbor block. According to a resultof the check, it is able to determine which neighbor block will be usedor which value will be used. It is checked whether flag information ofthe neighbor block is true or false. According to the check result, itis able to determine whether to use the neighbor block.

According to another embodiment of the present invention, a process forperforming illumination compensation based on a macroblock type isexplained as follows. A process for deriving an IC flag and ICdifference information may differ according to the macroblock type.

For instance, in case that a macroblock type is P_Skip, it is able topreferentially check whether neighbor blocks are available and whetherillumination compensation is performed. If all of the neighbor blocksare available and if the illumination compensation is performed, an ICflag and IC difference information of a current block can be derived inthe following manner. First of all, if an average of IC differenceinformations of the neighbor blocks is not 0, the IC flag of the currentblock is set to 1 and the IC difference information of the current blockcan be set to the average value of the IC difference informations of theneighbor blocks. For instance, the neighbor blocks can be right and leftblocks of the current block. Otherwise, if the average of the ICdifference informations of the neighbor blocks is 0, each of the IC flagof the current block and the IC difference information of the currentblock can be set to 0.

In case that a prescribed one of neighbor blocks is available andillumination compensation is performed, an IC flag of a current block isset to 1 and IC difference information of the current block can be setto IC difference information of the neighbor block. For instance, theneighbor block can be a left or upper block of the current block.Alternatively, the neighbor block can be an upper right block or anupper left block. In this case, in checking the prescribed one of theneighbor blocks, a left block can be checked after an upper block hasbeen checked.

In other cases, an IC flag and IC difference information of a currentblock can be set to 0 each. For example, the three kinds of casesincluding a case of using two blocks, upper and left blocks of a currentblock, a case of using an upper block of a current block only and a caseof using a left block of current block are excluded for the cases.

Meanwhile, in case that a macroblock type is B_Skip, IC differenceinformation of a current block can be derived in a following manner.First of all, it is able to determine whether a reference index of acurrent block is identical to that of a neighbor block. Based on thedetermination result, it is able to obtain a predicted value forillumination compensation of the current block. Using the obtainedpredicted value, it is able to reconstruct IC difference information ofthe current block. The step of determining whether the reference indexesof the current and neighbor blocks are identical to each other and thestep of obtaining the predicted value based on the determination resultare explained in detail as follows.

For example, it is able to determine whether a neighbor block having thereference index identical to that of a current block exists or not. Inthis case, it is assumed that a neighbor block to be used is a blockencoded using illumination compensation. As a result of thedetermination, when there exists a single neighbor block having thereference index identical to that of the current block, it is able toassign IC difference information of the neighbor block having theidentical reference index to an IC difference predictive value of thecurrent block. For instance, when an upper block of a current block is ablock that is encoded using illumination compensation and when the upperblock has the reference index identical to that of the current block, itis able to set IC difference information of the upper block to an ICdifference predictive value of the current block. Otherwise, the sameprocess can be carried out on an upper left block of the current block.Otherwise, the same process can be carried out on an upper right blockof the current block. Otherwise, if three neighbor blocks at upper, leftand upper right sides of the current block are blocks using illuminationcompensation, it is able to set an IC difference predictive value of thecurrent block to a median value between IC difference informations ofthe three neighbor blocks. Otherwise, it is able to set an IC differencepredictive value of the current block to 0.

If there exist two neighbor blocks having the reference index identicalto that of a current block, it is able to set an IC differencepredictive value of the current block to an average value between ICdifference informations of the two neighbor blocks.

The IC difference predictive value deriving methods is identicallyapplicable to a process for predicting an IC flag.

Meanwhile, in case that a macroblock type is B_Skip, an IC flag of acurrent block can be derived in the following manner. The IC flag of thecurrent block can be derived based on IC difference information of thecurrent block.

For instance, when IC difference information of a current block is 0, anIC flag, of the current block can be set to 0. Otherwise, an IC flag ofthe current block can be set to 1. In this case, the IC differenceinformation of the current block can be the value identical to an ICdifference predictive value that is set according to one of the variousembodiments for deriving the IC difference predictive value.

Meanwhile, when a macroblock type is B_Direct_(—)16×16, IC differenceinformation of a current block can be derived as follows. For instance,when a type of a current block is B_Direct_(—)16×16 and an IC flag ofthe current block is set to 1, IC difference information of the currentblock can be derived according to a sum of a transmitted IC differenceresidual value and a derived IC difference predictive value. In thiscase, the derived IC difference predictive value can be derivedaccording to the above-explained various embodiments.

A method of performing illumination compensation using flag informationindicating a presence or non-presence of an execution of theillumination compensation of a corresponding block according to anotherembodiment of the present invention is explained as follows.

When reconstructing IC difference information of a current block, it isable to use the aforesaid IC flag (IC_flag) indicating a presence ornon-presence of an execution of the illumination compensation of acorresponding block. Alternatively, it is able to obtain the ICdifference predictive value using both of the above-mentioned method ofchecking a reference index and the above-mentioned method of predictingan IC flag.

First of all, it is able to determine whether there exists a neighborblock having the reference index identical to that of a current block.Based on the determination result, it is able to obtain an IC differencepredictive value for illumination compensation of the current block. Inthis case, the IC difference predictive value can be obtained based onwhether an IC flag of a neighbor block is 1. And, it is able so able topredict an IC flag of the current block based on the result. Hence, itis able to perform the illumination compensation by reconstructing ICdifference information of the current block using the obtained ICdifference predictive value and the predicted IC flag.

A method of predicting an IC flag of a current block based on whether areference index of the current bock is identical to a reference index ofa neighbor block according to another embodiment of the presentinvention is explained as follows.

First of all, it is able to determine whether there exists a neighborblock having the reference index identical to that of a current block.As the determination result, when there exits a single neighbor blockhaving the reference index identical to that of the current bock, it isable to predict an IC flag of the current block from an IC flag of theneighbor block having the same reference index. According to thedetermination result, when there exist two neighbor blocks having thereference index identical to that of the current block, it is able topredict an IC flag of the current block from one of IC flags of the twoneighbor blocks having the same reference index. According to thedetermination result, when there exist three neighbor blocks having thereference index identical to that of the current block, it is able topredict an IC flag of the current block from a median value of IC flagsof the three neighbor blocks having the same reference index. when theredoes not exit a neighbor block having the reference index identical tothat of the current bock, the IC flag prediction of the current block isnot performed.

According to another embodiment of the present invention,context-modeling different from the case of 16×16 inter-mode, can beperformed to macroblock types, to which illumination compensation isapplied. For flag information, three contexts can be consideredaccording to flag values of neighbor blocks (e.g., left and upper blocksof current block). A case of a flag value ‘true’ is converted to 1 and acase of flag value ‘false’ converted to 0. If two values for therespective cases are summed, it results in three kinds of cases. So,flag information is encoded/decoded using these three contexts. Forexample, it is able to use two context models for IC residual, liketransform coefficient levels coding. In particular, binarization isexecuted by UEGO (unary/0th order Exp-Golomb), a single context model isapplied to a first bin value, and a single model context is applied tothe rest bin values of a unary prefix part. Sign bit can beencoded/decoded in bypass mode. As another embodiment of flaginformation, three contexts can be considered according to a value ofpredicted flag information. Using this, it is able to performencoding/decoding.

The above description in this disclosure is applicable to macroblockunit. And, the above description in this disclosure is also applicableto smaller blocks.

FIG. 6 is a diagram to explain a coding method using a depth mapaccording to an embodiment of the present invention.

In video signal coding, it is able to use depth information for aspecific application or a different purpose. The depth information maymean the information capable of indicating an inter-view disparitydifference. For instance, it is able to obtain a disparity vectorthrough inter-view prediction. And, the obtained disparity vector shouldbe transmitted to a decoding device for disparity compensation of acurrent block. Yet, if the depth map is transmitted to a decodingdevice, it is able to derive the disparity vector from the depth map (ora disparity map) without transmitting the disparity vector to thedecoding device. And, it is also able to transmit a motion vector or adisparity vector together with the depth map. In this case, the depthmap may mean that depth information is indicated for each predeterminedunit. For instance, the predetermined unit may correspond to a pixelunit or a block unit.

According to an embodiment of the present invention, a case of coding adepth map together with color components is explained as follows.

First of all, information indicating whether a depth map is currentlyused for a vide sequence. For instance, it is able to obtaindepth_coding_flag from an extension area of a sequence parameter set. Ifthe depth_coding_flag is 0, it is able to code color components, e.g.,YUV components only. If the depth_coding_flag is 1, a depth map is codedtogether with the color components and then used.

According to another embodiment of the present invention, variousschemes are applicable in using a depth map. For instance, various kindsof depth maps are usable according to spatial resolution. In particular,if depth_map_mode is 0, it may mean that a depth map is not used. Ifdepth_map_mode is 1, it may mean that a depth map of full resolution isused. If depth_map_mode is 2, it may mean that a depth map ½ downsampledin horizontal direction is used. If depth_map_mode is 3, it may meanthat a depth map ½ downsampled in vertical direction is used. Thesesvalues are just embodiments and other values can be set up. Moreover,depth maps of various spatial resolutions are usable as well as ½downsampling.

Meanwhile, in case that an inputted depth map has a full resolution, ifdepth_map_mode indicates a depth map of a different resolution, adownsampling process should be performed by an encoder and an upsamplingprocess should be performed by a decoder. So, if an inputted depth map,as shown in FIG. 5, is already downsampled with a low resolution, adecoder may perform an upsampling process with a full resolution.Moreover, it is able to reconstruct a current picture using offsetvalues indicating position difference between a color picture and anupsampled depth picture. The offset may include a left offset, a rightoffset, a top offset, and a bottom offset.

The depth_coding_flag and the depth_mode_map can be obtained from asequence parameter set, an extension area of a sub-sequence parameterset, a picture parameter set, a slice header or a macroblock layer.

FIGS. 7 to 14 are diagrams of syntaxes for describing variousapplication examples that use camera information according to anembodiment of the present invention.

First of all, camera information is the information about a camera thatgenerates a sequence corresponding to a series of pictures. A series ofpictures captured by a single camera construct a single view. Even if aposition of a camera is changed, it is able to construct a new view.Meanwhile, as a type of camera information, there can be a cameraparameter. The camera parameters can be classified into an intrinsiccamera parameter and an extrinsic camera parameter. The intrinsic cameraparameters can include focal length, aspect ratio, skew, principal pointand the like. And, the extrinsic camera parameters can include positioninformation of a camera in world coordinate system, rotation matrix,translation vector and the like. And, it is able to precisely findgeometrical relations between cameras using informations included in theextrinsic camera parameters.

And, it is able to utilize topological relations between cameras. Forinstance, if camera arrangement makes 1-dimensional orhorizontal/vertical 2-dimensional configuration, it is able to observethe topological relation using information on the camera arrangement.Hence, a intuitively specified view can be selected and then displayed.In doing so, the information on the camera arrangement can betransmitted as side information.

For instance, the information on the camera arrangement can correspondto a linear or arc configuration according to a presence or non-presenceof linearity or a 1- or 2-dimensional configuration according to adimension. In case that the camera arrangement corresponds to a2-dimensional arrangement, there can exist the same number of views perrow according to a presence or non-presence of normality or thedifferent number of views per row. Thus, as various references areapplied, a transmitted syntax structure may vary. For this, detailedembodiments are explained as follows.

First of all, it is able to define a flag indicating a prescribedconfiguration of camera arrangement. For instance, iflinear_configuration_flag is 1, it may mean that camera arrangement islinear. If linear_configuration_flag is 0, it may mean that cameraarrangement is arc. Besides, the flag can include various types and canbe changed according to definitions.

Assuming that camera arrangement corresponds to a 2-dimensionalconfiguration, it is able to define a flag indicating whether a cameraactually exists at each node. For instance, if camera_present_flag[i][j]is 1, it may mean that a camera exists at a node in i^(th) row andj^(th) column. If camera_present_flag[i][j] is 0, it may mean that acamera does not exist at a node in the i^(th) row and j^(th) column.

Assuming that camera arrangement corresponds to a 2-dimensionalconfiguration, it is able to define information indicating the numbersof maximum views in horizontal and vertical directions, respectively.For instance, max_num_view_hor_minus1 may mean the maximum view number 1in horizontal direction. And, max_num_view_ver_minus1 may mean themaximum view number −1 in vertical direction. Moreover,num_view_hor_minus1[i] may mean the maximum view number −1 in horizontaldirection of a j^(th) column.

And, view_id[i][j] may mean a view identification number at a node ini^(th) row and j^(th) column.

It is able to define information indicating a prescribed dimension ofcamera arrangement. For instance, if one_dimension_flag is 1, it maymean that cameras are arranged in horizontal direction only. Ifone_dimension_flag is 0, it may mean that cameras are 2-dimensaionallyarranged in vertical and horizontal directions. Besides, the flag caninclude various dimensions and may vary according to how to be defined.For instance, if regular_two_dimension_flag is 1, it may mean that thenumber of views on each row is identical in horizontal or verticaldirection. If regular_two_dimension_flag is 0, it may mean that thenumber of views on each row is not identical in horizontal or verticaldirection.

The above-described informations can generate various applicationexamples by being combined into various shaped. Several embodiments areexplained in the following description.

In the embodiment shown in FIG. 7, shown is a syntax structureindicating whether a camera is present at each node in 2-dimensionalcamera arrangement. First of all, it is able to check whether camerasare arranged in linear or arc configuration according to information oflinear_configuration_flag. And, it is able to obtain the number ofmaximum views in horizontal/vertical direction. According tocamera_present_flag[i][j], it is able to check whether camera is presentat each node. If the camera is present, it is able to know a viewidentification number at a position whether the camera is present.

FIG. 8 shows a syntax structure facilitating camera arrangement to bechecked in case of 1-dimensional configuration. Unlike FIG. 7, FIG. 8shows the 1-dimensional configuration, it is not necessary to check apresence or non-presence of camera at each node. Hence, it is able toknow camera arrangement by checking a view identification numberdirectly. Below the else syntax, it is able to confirm 2-dimensionalcamera arrangement like FIG. 6.

FIG. 9 shows a syntax structure enabling camera arrangement to bechecked appropriately in accordance with a flag indicating a type ofdimensional arrangement. First of all, in case that a 2-dimensionalconfiguration is indicated according to one_dimension_flag, it is ableto check each view identification number by obtaining the number ofviews on each column in vertical direction. Meanwhile, in case that a1-dimensional configuration is indicated, it is able to check each viewidentification number by obtaining the number of maximum views inhorizontal direction.

FIG. 10 shows an embodiment resulting from combining the embodiments ofFIG. 7 and FIG. 9 together. In this embodiment, it is able to check apresence or non-presence of camera according to one_dimension_flagindicating a prescribed dimensional arrangement.

In an embodiment shown in FIG. 11, a view identification number isdirectly checked using the maximum view number in horizontal directionin case of a 1-dimensional arrangement. In case of a 2-dimensionalarrangement, a view identification number is checked using the maximumview number in vertical direction additionally.

In an embodiment shown in FIG. 12, it is able to check cameraarrangement using information indicating whether the number of views oneach column in horizontal or vertical direction is identical. First ofall, if the number of views on each column in horizontal or verticaldirection is not identical, it is able to check camera arrangement bychecking the view number in horizontal direction on each column invertical direction. On the other hand, in case that the number of viewson each column in horizontal or vertical direction is identical, it isable to check each view identification information according to thenumber of maximum views in horizontal/vertical direction.

FIG. 13 shows an embodiment resulting from combining the embodiments ofFIG. 12 and FIG. 7 together. First of all, if the number of views oneach column in horizontal or vertical direction is not identical, it isable to check a presence or non-presence of camera according to thenumber of maximum views in horizontal/vertical direction. On the otherhand, if the number of views on each column in horizontal or verticaldirection is identical, it is able to check each view identificationnumber according to the number of maximum views in horizontal/verticaldirection.

FIG. 14 shows an embodiment resulting from combining the embodiments ofFIG. 12 and FIG. 9 together. First of all, only in case of a2-dimensional arrangement according to a flag indicating prescribeddimension(s) of arrangement, it is able to obtain information indicatingwhether the number of views on each column in horizontal or verticaldirection is identical. And, it is also able to obtain informationindicating the number of maximum views in vertical direction. Otherwise,each of the two informations can be set to 0. The rest of the syntax isexplained in the former description of the embodiment shown in FIG. 12.

FIG. 15 is a diagram of an overall prediction structure of a multi-viewsequence signal according to an embodiment of the present invention toexplain a concept of an inter-view picture group.

Referring to FIG. 15, T0 to T100 on a horizontal axis indicate framesaccording to time and S0 to S7 on a vertical axis indicate framesaccording to view. For instance, pictures at T0 mean sequences capturedby different cameras on the same time zone T0, while pictures at S0 meansequences captured by a single camera on different time zones. And,arrows in the drawing indicate predicted directions and orders of therespective pictures. For instance, a picture P0 in a view S2 on a timezone T0 is a picture predicted from I0, which becomes a referencepicture of a picture P0 in a view S4 on the time zone T0. And, itbecomes a reference picture of pictures B1 and B2 on time zones T4 andT2 in the view S2, respectively.

For a multi-view sequence decoding process, an inter-view random accessmay be required. So, an access to a random view should be possible byminimizing the decoding process. In this case, a concept of aninter-view picture group may be needed to perform an efficient randomaccess. The definition of the inter-view picture group was mentioned inFIG. 2. For instance, in FIG. 3, if a picture I0 in a view S0 on a timezone T0 corresponds to an inter-view picture group, all pictures indifferent views on the same time zone, i.e., the time zone T0 cancorrespond to the inter-view picture group. For another instance, if apicture I0 in a view S0 on a time zone T8 corresponds to an inter-viewpicture group, all pictures in different views on the same time zone,i.e., the time zone T8 can correspond to the inter-view picture group.Likewise, all pictures in T16, . . . , T96, and T100 become an exampleof the inter-view picture group as well. According to anotherembodiment, in an overall prediction structure of MVC, GOP can startfrom a I picture. And, the I picture is compatible with H.264/AVC. So,all inter-view picture groups compatible with H.264/AVC can become the Ipicture. Yet, in case of replacing the pictures-I by P picture, moreefficient coding is possible. In particular, more efficient coding isenabled using a prediction structure that GOP is made to start from Ppicture compatible with H.264/AVC.

In this case, if the inter-view picture group is re-defined, it becomesa coded picture capable of referring to a slice on a different time zonein a same view as well as a slice that all slices exist in a frame on asame time zone. Yet, the case of referring to a slice on a differenttime zone in a same view may be limited to an inter-view picture groupcompatible with H.264/AVC only.

After the inter-view picture group has been decoded, all of thesequentially coded pictures are decoded from the picture decoded aheadof the inter-view picture group in an output order withoutinter-prediction.

Considering the overall coding structure of the multi-view videosequence shown in FIG. 15, since inter-view dependency of an inter-viewpicture group differs from that of a non-inter-view picture group, it isnecessary to discriminate the inter-view picture group and thenon-inter-view picture group from each other according to the inter-viewpicture group identification information.

The inter-view reference information means information indicating whatkind of structure is used to predict inter-view sequences. This can beobtained from a data area of a video signal. For instance, it can beobtained from a sequence parameter set area. And, the inter-viewreference information can be obtained using the number of referencepictures and view information of the reference pictures. For instance,after a total number of views has been obtained, it is able to obtainview identification information for identifying each view based on thetotal number of the views. And, number information of inter-viewreference pictures, which indicates a number of reference pictures for areference direction of each view, can be obtained. According to thenumber information of the inter-view reference pictures, it is able toobtain view identification information of each inter-view referencepicture.

Through this method, the inter-view reference information can beobtained. And, the inter-view reference information can be obtained in amanner of being categorized into a case of an inter-view picture groupand a case of a non-inter-view picture group. This can be known usinginter-view picture group identification information indicating whether acoded slice in a current NAL corresponds to an inter-view picture group.The inter-view picture group identification information can be obtainedfrom an extension area of NAL header or a slice layer area.

The inter-view reference information obtained according to theinter-view picture group identification information is usable forconstruction, management and the like of a reference picture list.

In the following description, various methods for indicating inter-viewreference information are explained.

FIGS. 16 to 20 are diagrams of various syntaxes for describinginter-view reference information according to an embodiment of thepresent invention.

As mentioned in the foregoing description of FIG. 16, inter-viewreference information can be independently understood by beingclassified into a case of an inter-view picture group and a case of anon-inter-view picture group.

FIG. 16 shows a syntax structure capable eliminating data that isredundant in obtaining inter-view reference information.

Referring to FIG. 16, it is able to obtain view identificationinformation of each inter-view reference picture according to the totalview number. In this case, a view having view_id[0] may correspond to abase view or an independent view. If so, in the number information ofinter-view reference picture, the number in a L0 direction becomes 0that is equal to that in a L1 direction. This is applicable to the caseof the inter-view or non-inter-view picture group in the same manner.For instance, in case of the inter-view picture group,‘num_anchor_refs_l0[0]=num_anchor_refs_l1[0]=0’ can be established[S1620, S1630]. In case of the non-inter-view picture group,‘num_non_anchor_refs_l0[0]=num_non_anchor_refs_l1[0]=0’ can beestablished [S1650, S1660]. Hence, if i=0, the redundant data may not betransmitted [S1610, S1640]. And, the number information of inter-viewreference pictures in L0/L1 direction in case of ‘i=0’ can be set to 0each. This is applicable to the case of the inter-view or non-inter-viewpicture group in the same manner.

According to another embodiment of the present invention, it is able todefine view identification information to have a different meaning. Forinstance, view identification information may mean a view identificationnumber assigned to each view according to a coding order. Alternatively,the view identification information (view_id[i]) may mean a viewidentification number assigned to each view according to a random order.Alternatively, the view identification information (view_id[i]) may meana view identification number assigned to each view according to cameraarrangement information.

FIG. 17 shows a syntax indicating inter-view reference information usingsimilarity between a prediction structure of an inter-view picture groupand a prediction structure of a non-inter-view picture group accordingto an embodiment of the present invention.

The prediction structure of the non-inter-view picture group may havethe structure dependent on the prediction structure of the inter-viewpicture group. So, the inter-view reference relation of the inter-viewpicture group can be indicated using the inter-view reference relationof the non-inter-view picture group.

Referring to FIG. 17, it is able to obtain inter-view referenceinformation of a non-inter-view picture group pS1710]. For instance, itis able to obtain num_non_anchor_refs_l0[i], non_anchor_refs_l0[i][j],num_non_anchor_refs_l1[i], and non_anchor_refs_l1[i][j].

And, the following ‘for’ statement indicates an inter-view referencerelation of the inter-view picture group. In this case, it is able touse the obtained inter-view reference information on the non-inter-viewpicture group. First of all, by setting a value ofnon_anchor_refs_l0[i][j] for anchor_refs_l0[i][j], it is able to know areference relation of the non-inter-view reference group [S1720]. Byobtaining anchor_refs_l0[i][j] as many as a number difference betweeninter-view reference pictures and non-inter-view reference pictures, itis able to understand the rest predictive structure of the inter-viewpicture group [S1730]. This is applicable to the L1 direction in thesame manner [S1740, S1750].

If so, the following relation shown in Formula 6 can be established.num_anchor_refs_(—) l0[i]≧num_non_anchor_refs_(—) l0[i]num_anchor_refs_(—) l1[i]≧num_non_anchor_refs_(—) l1[i]  [Formula 6]

When the inter-view reference relation is represented, the predictionstructure of the inter-view picture group can representatively indicatea prediction structure of an overall sequence. If information necessaryfor the prediction relation between non-inter-view picture groups isderived from inter-view picture groups, more efficient coding isenabled. For instance, in case of motion skip mode, it is able to use aprediction structure of inter-view picture group when a current block ina non-inter-view picture group uses motion information of a picture in aneighbor view. In particular, it is able to use motion informationobtained from a corresponding picture in a prediction structure ofinter-view picture group.

FIG. 18 shows a syntax indicating inter-view reference information usingsimilarity between a prediction structure of an inter-view picture groupand a prediction structure of a non-inter-view picture group accordingto an embodiment of the present invention.

The syntax structure shown in FIG. 18 is similar to that shown in FIG.17. Yet, in FIG. 18, it is able to obtain information(num_diff_anchor_non_anchor_refs_l0[i],num_diff_anchor_non_anchor_refs_l1[i]) indicating a difference betweenthe number of inter-view reference pictures and the number ofnon-inter-view reference pictures [S1810, S1830]. By obtaininganchor_refs_l0[i][j] using the information indicating the differencebetween the number of inter-view reference pictures and the number ofnon-inter-view reference pictures, it is able to understand the restprediction structure of the inter-view picture group [S1820, S1840]. Inthis case, the rest prediction structure may mean a prediction structureof inter-view picture group only different from a structure ofnon-inter-view picture group.

FIG. 19 shows a syntax structure to indicate inter-view referencerelation in a simpler manner according to an embodiment of the presentinvention.

If a prediction structure of non-inter-view picture group becomes asubset of a prediction structure of inter-view picture group, it may bemore effective. This is because inter-view prediction in view directionis possible in inter-view picture group only but temporal prediction intime direction is impossible. Hence, it is referable that inter-viewprediction is maximally utilized in aspect of eliminating redundancy ofinter-view sequence. On the other hand, in non-inter-view picture group,both view-directional prediction and time-directional prediction arepossible. Yet, coding gain through the time-directional prediction maybe greater than that through the view-directional prediction. Hence, itmay be a more effective structure that the inter-view predictionrelation in the non-inter-view picture group belongs to the inter-viewprediction structure in the inter-view picture group.

Thus, the prediction structure of the non-inter-view picture group mayhave the structure dependent on the prediction structure of theinter-view picture group. So, the inter-view reference relation of theinter-view picture group can be represented using the inter-viewreference relation of the non-inter-view picture group. This may beefficient in aspects of random access and view scalability as well. Forinstance, in case of a picture that belongs not to a predictionstructure of non-inter-view picture group but to a prediction structureof inter-view picture group, it may be efficient because the pictureneeds not to be decoded.

Meanwhile, there may be a case that there exists no inter-viewprediction relation in inter-view or non-inter-view picture group. Thus,the absence of the inter-view prediction relation in the inter-viewpicture group may be more effective to fast random access and parallelprocessing. And, it may be introduced to the non-inter-view picturegroup to reduce complexity.

Hence, according to an embodiment of the present invention, it is ableto represent inter-view reference relation in a single format‘multiview_ref_lx’ instead of representing the inter-view referencerelation by discriminating anchor_ref_lX and non_anchor_ref_lX [S1920.S1940].

For instance, in case that inter-view prediction relation in inter-viewpicture group is absent, in order to include a case ofnum_anchor_refs_l0[i]=num_anchor_refs_l1[i]=0, Formula 7 is usable[S1910, S1930].max(num_anchor_refs_(—) lx[i],num_non_anchor_refs_(—) lx[i])  [Formula7]

In this case, an inter-view reference in i^(th) view can be representedas a single syntax (multiview_ref_lx[i]). Moreover, ‘from 0 tomin(num_anchor_refs_lx[i], num_non_anchor_refs_lx[i]−1)’ is able torepresent a reference relation common to inter-view picture group andnon-inter-view picture group.

FIG. 20 shows a syntax structure capable of eliminating data that isredundant in obtaining inter-view reference information.

In FIG. 19, it is able to obtain view identification information ofinter-view reference picture according to a value ofmax(num_anchor_refs_lx[i], num_non_anchor_refs_lx[i]). In this case, aview having view_id[0] may correspond to a base view or an independentview. If so, in the number information of inter-view reference pictures,the number in a L0 direction becomes 0 that is equal to that in a L1direction. This is applicable to the case of the inter-view ornon-inter-view picture group in the same manner. For instance, in caseof the inter-view picture group,‘num_anchor_refs_l0[0]=num_anchor_refs_l1[0]=0’ can be established. Incase of the non-inter-view picture group,‘num_non_anchor_refs_l0[0]=num_non_anchor_refs_l1[0]=0’ can beestablished. Hence, if i=0, the redundant data may not be transmitted[S2010]. And, the number information of inter-view reference pictures inL0/L1 direction in case of ‘i=0’ can be set to 0 each.

As mentioned in the foregoing description, the decoding/encoding device,to which the present invention is applied, is provided to atransmitter/receiver for multimedia broadcasting such as DMB (digitalmultimedia broadcast) to be used in decoding video and data signals andthe like. And, the multimedia broadcast transmitter/receiver can includea mobile communication terminal.

A decoding/encoding method, to which the present invention is applied,is configured with a program for computer execution and then stored in acomputer-readable recording medium. And, multimedia data having a datastructure of the present invention can be stored in computer-readablerecording medium. The computer-readable recording media include allkinds of storage devices for storing data that can be read by a computersystem. The computer-readable recording media include ROM, RAM, CD-ROM,magnetic tapes, floppy discs, optical data storage devices, etc. andalso includes a device implemented with carrier waves (e.g.,transmission via internet). And, a bit stream generated by the encodingmethod is stored in a computer-readable recording medium or transmittedvia wire/wireless communication network.

INDUSTRIAL APPLICABILITY

Accordingly, while the present invention has been described andillustrated herein with reference to the preferred embodiments thereof,it will be apparent to those skilled in the art that variousmodifications and variations can be made therein without departing fromthe spirit and scope of the invention. Thus, it is intended that thepresent invention covers the modifications and variations of thisinvention that come within the scope of the appended claims and theirequivalents.

What is claimed is:
 1. A method of decoding a video signal with adecoding apparatus, comprising: obtaining, with the decoding apparatus,Illumination Compensation difference information of a neighbor block ofa current block, when the video signal does not include an IlluminationCompensation flag for the current block; deriving, with the decodingapparatus, an Illumination Compensation difference predictor of thecurrent block using the Illumination Compensation difference informationof the neighbor block; deriving, with the decoding apparatus, theIllumination Compensation flag based on the Illumination Compensationdifference predictor of the current block, the Illumination Compensationflag indicating whether illumination compensation is performed on thecurrent block; and performing, with the decoding apparatus, theillumination compensation on the current block, based on theIllumination Compensation difference predictor of the current block andthe Illumination Compensation flag, wherein when the IlluminationCompensation difference predictor of the current block is 0, theIllumination Compensation flag of the current block is derived as 0, andwhen the Illumination Compensation difference predictor of the currentblock is not 0, the Illumination Compensation flag of the current blockis derived as
 1. 2. The method of claim 1, wherein the IlluminationCompensation difference predictor of the current block is derived basedon whether a reference index of the current block is identical to thatof the neighbor block.
 3. The method of claim 2, wherein when thereexists one neighbor block having the reference index identical to thatof the current block, the Illumination Compensation difference predictorof the current block is set to the Illumination Compensation differenceinformation of the neighbor block.
 4. The method of claim 3, wherein theneighbor block is checked in order of upper, left, upper-right andupper-left blocks of the current block.
 5. The method of claim 2,wherein when there exists three neighbor blocks, each having thereference index identical to that of the current block, the IlluminationCompensation difference predictor of the current block is set to amedian value of Illumination Compensation difference information of thethree neighbor blocks.
 6. The method of claim 1, wherein the videosignal is received as a broadcast signal.
 7. The method of claim 1,wherein the video signal is received via digital medium.
 8. An apparatusfor decoding a video signal, comprising: a decoding apparatus configuredto derive an Illumination Compensation difference predictor of a currentblock using obtained Illumination Compensation difference information ofa neighbor block, when information associated with illuminationcompensation for the current block is not available in a bitstream, thedecoding apparatus configured to derive an Illumination Compensationflag based on the Illumination Compensation difference predictor of thecurrent block, and configured to perform the illumination compensationon the current block based on the Illumination Compensation differencepredictor of the current block and the Illumination Compensation flag,wherein when the Illumination Compensation difference predictor of thecurrent block is 0, the Illumination Compensation flag of the currentblock is derived as 0, and when the Illumination Compensation differencepredictor of the current block is not 0, the Illumination Compensationflag of the current block is derived as 1.